Scale-Space Hypernetworks for Efficient Biomedical Imaging

Convolutional Neural Networks (CNNs) are the predominant model used for a variety of medical image analysis tasks. At inference time, these models are computationally intensive, especially with volumetric data. In principle, it is possible to trade accuracy for computational efficiency by manipulating the rescaling factor in the downsample and upsample layers of CNN architectures. However, properly exploring the accuracy-efficiency trade-off is prohibitively expensive with existing models. To address this, we introduce Scale-Space HyperNetworks (SSHN), a method that learns a spectrum of CNNs with varying internal rescaling factors. A single SSHN characterizes an entire Pareto accuracy-efficiency curve of models that match, and occasionally surpass, the outcomes of training many separate networks with fixed rescaling factors. We demonstrate the proposed approach in several medical image analysis applications, comparing SSHN against strategies with both fixed and dynamic rescaling factors. We find that SSHN consistently provides a better accuracy-efficiency trade-off at a fraction of the training cost. Trained SSHNs enable the user to quickly choose a rescaling factor that appropriately balances accuracy and computational efficiency for their particular needs at inference.Comment: Code available at https://github.com/JJGO/scale-space-hypernetwork

Magnitude Invariant Parametrizations Improve Hypernetwork Learning

Hypernetworks, neural networks that predict the parameters of another neural network, are powerful models that have been successfully used in diverse applications from image generation to multi-task learning. Unfortunately, existing hypernetworks are often challenging to train. Training typically converges far more slowly than for non-hypernetwork models, and the rate of convergence can be very sensitive to hyperparameter choices. In this work, we identify a fundamental and previously unidentified problem that contributes to the challenge of training hypernetworks: a magnitude proportionality between the inputs and outputs of the hypernetwork. We demonstrate both analytically and empirically that this can lead to unstable optimization, thereby slowing down convergence, and sometimes even preventing any learning. We present a simple solution to this problem using a revised hypernetwork formulation that we call Magnitude Invariant Parametrizations (MIP). We demonstrate the proposed solution on several hypernetwork tasks, where it consistently stabilizes training and achieves faster convergence. Furthermore, we perform a comprehensive ablation study including choices of activation function, normalization strategies, input dimensionality, and hypernetwork architecture; and find that MIP improves training in all scenarios. We provide easy-to-use code that can turn existing networks into MIP-based hypernetworks.Comment: Source code at https://github.com/JJGO/hyperligh

UniverSeg: Universal Medical Image Segmentation

While deep learning models have become the predominant method for medical image segmentation, they are typically not capable of generalizing to unseen segmentation tasks involving new anatomies, image modalities, or labels. Given a new segmentation task, researchers generally have to train or fine-tune models, which is time-consuming and poses a substantial barrier for clinical researchers, who often lack the resources and expertise to train neural networks. We present UniverSeg, a method for solving unseen medical segmentation tasks without additional training. Given a query image and example set of image-label pairs that define a new segmentation task, UniverSeg employs a new Cross-Block mechanism to produce accurate segmentation maps without the need for additional training. To achieve generalization to new tasks, we have gathered and standardized a collection of 53 open-access medical segmentation datasets with over 22,000 scans, which we refer to as MegaMedical. We used this collection to train UniverSeg on a diverse set of anatomies and imaging modalities. We demonstrate that UniverSeg substantially outperforms several related methods on unseen tasks, and thoroughly analyze and draw insights about important aspects of the proposed system. The UniverSeg source code and model weights are freely available at https://universeg.csail.mit.eduComment: Victor and Jose Javier contributed equally to this work. Project Website: https://universeg.csail.mit.ed

Subclinical Atherosclerosis Burden by 3D Ultrasound in Mid-Life: The PESA Study

BACKGROUND: Detection of subclinical atherosclerosis improves risk prediction beyond cardiovascular risk factors (CVRFs) and risk scores, but quantification of plaque burden may improve it further. Novel 3-dimensional vascular ultrasound (3DVUS) provides accurate volumetric quantification of plaque burden. OBJECTIVES: The authors evaluated associations between 3DVUS-based plaque burden and CVRFs and explored potential added value over simple plaque detection. METHODS: The authors included 3,860 (92.2%) PESA (Progression of Early Subclinical Atherosclerosis) study participants (age 45.8 ± 4.3 years; 63% men). Bilateral carotid and femoral territories were explored by 3DVUS to determine the number of plaques and territories affected, and to quantify global plaque burden defined as the sum of all plaque volumes. Linear regression and proportional odds models were used to evaluate associations of plaque burden with CVRFs and estimated 10-year cardiovascular risk. RESULTS: Plaque burden was higher in men (63.4 mm3 [interquartile range (IQR): 23.8 to 144.8 mm3] vs. 25.7 mm3 [IQR: 11.5 to 61.6 mm3] in women; p < 0.001), in the femoral territory (64 mm3 [IQR: 27.6 to 140.5 mm3] vs. 23.1 mm3 [IQR: 9.9 to 48.7 mm3] in the carotid territory; p < 0.001), and with increasing age (p < 0.001). Age, sex, smoking, and dyslipidemia were more strongly associated with femoral than with carotid disease burden, whereas hypertension and diabetes showed no territorial differences. Plaque burden was directly associated with estimated cardiovascular risk independently of the number of plaques or territories affected (p < 0.01). CONCLUSIONS: 3DVUS quantifies higher plaque burden in men, in the femoral territory, and with increasing age during midlife. Plaque burden correlates strongly with CVRFs, especially at the femoral level, and reflects estimated cardiovascular risk more closely than plaque detection alone. (Progression of Early Subclinical Atherosclerosis [PESA] Study; NCT01410318).The PESA study is cofunded equally by the Fundacion Centro Nacional de Investigaciones Cardiovasculares Carlos III (CNIC), Madrid, Spain, and Banco Santander, Madrid, Spain. The study also receives funding from the Institute of Health Carlos III (PI15/02019) and the European Regional Development Fund (ERDF). The CNIC is supported by the Ministry of Economy, Industry and Competitiveness (MINECO) and the Pro CNIC Foundation, and is a Severo Ochoa Center of Excellence (MINECO award SEV-2015-0505). Dr. Sanchez-Gonzalez is an employee of Philips Healthcare. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose. Stephen J. Nicholls, MD, served as Guest Editor for this paperS

Vascular Inflammation in Subclinical Atherosclerosis Detected by Hybrid PET/MRI

BACKGROUND: Atherosclerosis is a chronic inflammatory disease, but data on arterial inflammation at early stages is limited. OBJECTIVES: The purpose of this study was to characterize vascular inflammation by hybrid 18F-fluorodeoxyglucose (18F-FDG) positron emission tomography/magnetic resonance imaging (PET/MRI). METHODS: Carotid, aortic, and ilio-femoral 18F-FDG PET/MRI was performed in 755 individuals (age 40 to 54 years; 83.7% men) with known plaques detected by 2-/3-dimensional vascular ultrasound and/or coronary calcification in the PESA (Progression of Early Subclinical Atherosclerosis) study. The authors evaluated the presence, distribution, and number of arterial inflammatory foci (increased 18F-FDG uptake) and plaques with or without inflammation (coincident 18F-FDG uptake). RESULTS: Arterial inflammation was present in 48.2% of individuals (24.4% femorals, 19.3% aorta, 15.8% carotids, and 9.3% iliacs) and plaques in 90.1% (73.9% femorals, 55.8% iliacs, and 53.1% carotids). 18F-FDG arterial uptakes and plaques significantly increased with cardiovascular risk factors (p < 0.01). Coincident 18F-FDG uptakes were present in 287 of 2,605 (11%) plaques, and most uptakes were detected in plaque-free arterial segments (459 of 746; 61.5%). Plaque burden, defined by plaque presence, number, and volume, was significantly higher in individuals with arterial inflammation than in those without (p < 0.01). The number of plaques and 18F-FDG uptakes showed a positive albeit weak correlation (r = 0.25; p < 0.001). CONCLUSIONS: Arterial inflammation is highly prevalent in middle-aged individuals with known subclinical atherosclerosis. Large-scale multiterritorial PET/MRI allows characterization of atherosclerosis-related arterial inflammation and demonstrates 18F-FDG uptake in plaque-free arterial segments and, less frequently, within plaques. These findings suggest an arterial inflammatory state at early stages of atherosclerosis. (Progression of Early Subclinical Atherosclerosis [PESA]; NCT01410318).The PESA study is cofunded equally by the Centro Nacional de Investigaciones Cardiovasculares (CNIC) and Banco Santander. The study also receives funding from the Instituto de Salud Carlos III (PI15/02019) and the European Regional Development Fund (ERDF) “A way to make Europe.” The CNIC is supported by the Ministerio de Ciencia, Innovación y Universidades, and the Pro CNIC Foundation, and is a Severo Ochoa Center of Excellence (SEV-2015-0505). Dr. Sanchez-González is an employee of Philips Healthcare. Dr. Bueno has received research funding from the Instituto de Salud Carlos III, Spain (PIE16/00021 & PI17/01799), AstraZeneca, Bristol-Myers Squibb, Janssen, and Novartis; has received consulting fees from AstraZeneca, Bayer, Bristol-Myers Squibb-Pfizer, and Novartis; and has received speaking fees or support for attending scientific meetings from AstraZeneca, Bayer, Bristol-Myers Squibb-Pfizer, Novartis, and MEDSCAPE-the heart.org.S

Accurate quantification of atherosclerotic plaque volume by 3D vascular ultrasound using the volumetric linear array method.

Direct quantification of atherosclerotic plaque volume by three-dimensional vascular ultrasound (3DVUS) is more reproducible than 2DUS-based three-dimensional (2D/3D) techniques that generate pseudo-3D volumes from summed 2D plaque areas; however, its accuracy has not been reported. We aimed to determine 3DVUS accuracy for plaque volume measurement with special emphasis on small plaques (a hallmark of early atherosclerosis). The in vitro study consisted of nine phantoms of different volumes (small and medium-large) embedded at variable distances from the surface (superficial vs. >5 cm-depth) and comparison of 3DVUS data generated using a novel volumetric-linear array method with the real phantom volumes. The in vivo study was undertaken in a rabbit model of atherosclerosis in which 3DVUS and 2D/3D volume measurements were correlated against gold-standard histological measurements. In the in vitro setting, there was a strong correlation between 3DVUS measures and real phantom volume both for small (3.0-64.5 mm(3) size) and medium-large (91.1-965.5 mm(3) size) phantoms embedded superficially, with intraclass correlation coefficients (ICC) of 0.99 and 0.98, respectively; conversely, when phantoms were placed at >5 cm, the correlation was only moderate (ICC = 0.67). In the in vivo setting there was strong correlation between 3DVUS-measured plaque volumes and the histological gold-standard (ICC = 0.99 [4.02-92.5 mm(3) size]). Conversely, the correlation between 2D/3D values and the histological gold standard (sum of plaque areas) was weaker (ICC = 0.87 [49-520 mm(2) size]), with large dispersion of the differences between measurements in Bland-Altman plots (mean error, 79.2 mm(2)). 3DVUS using the volumetric-linear array method accurately measures plaque volumes, including those of small plaques. Measurements are more accurate for superficial arterial territories than for deep territories.S

Analysis of TNFAIP3, a feedback inhibitor of nuclear factor-κB and the neighbor intergenic 6q23 region in rheumatoid arthritis susceptibility

Introduction Genome-wide association studies of rheumatoid arthritis (RA) have identified an association of the disease with a 6q23 region devoid of genes. TNFAIP3, an RA candidate gene, flanks this region, and polymorphisms in both the TNFAIP3 gene and the intergenic region are associated with systemic lupus erythematosus. We hypothesized that there is a similar association with RA, including polymorphisms in TNFAIP3 and the intergenic region. Methods To test this hypothesis, we selected tag-single nucleotide polymorphisms (SNPs) in both loci. They were analyzed in 1,651 patients with RA and 1,619 control individuals of Spanish ancestry. Results Weak evidence of association was found both in the 6q23 intergenic region and in the TNFAIP3 locus. The rs582757 SNP and a common haplotype in the TNFAIP3 locus exhibited association with RA. In the intergenic region, two SNPs were associated, namely rs609438 and rs13207033. The latter was only associated in patients with anti-citrullinated peptide antibodies. Overall, statistical association was best explained by the interdependent contribution of SNPs from the two loci TNFAIP3 and the 6q23 intergenic region. Conclusions Our data are consistent with the hypothesis that several RA genetic factors exist in the 6q23 region, including polymorphisms in the TNFAIP3 gene, like that previously described for systemic lupus erythematosus

Bone marrow activation in response to metabolic syndrome and early atherosclerosis.

Experimental studies suggest that increased bone marrow (BM) activity is involved in the association between cardiovascular risk factors and inflammation in atherosclerosis. However, human data to support this association are sparse. The purpose was to study the association between cardiovascular risk factors, BM activation, and subclinical atherosclerosis. Whole body vascular 18F-fluorodeoxyglucose positron emission tomography/magnetic resonance imaging (18F-FDG PET/MRI) was performed in 745 apparently healthy individuals [median age 50.5 (46.8-53.6) years, 83.8% men] from the Progression of Early Subclinical Atherosclerosis (PESA) study. Bone marrow activation (defined as BM 18F-FDG uptake above the median maximal standardized uptake value) was assessed in the lumbar vertebrae (L3-L4). Systemic inflammation was indexed from circulating biomarkers. Early atherosclerosis was evaluated by arterial metabolic activity by 18F-FDG uptake in five vascular territories. Late atherosclerosis was evaluated by fully formed plaques on MRI. Subjects with BM activation were more frequently men (87.6 vs. 80.0%, P = 0.005) and more frequently had metabolic syndrome (MetS) (22.2 vs. 6.7%, P < 0.001). Bone marrow activation was significantly associated with all MetS components. Bone marrow activation was also associated with increased haematopoiesis-characterized by significantly elevated leucocyte (mainly neutrophil and monocytes) and erythrocyte counts-and with markers of systemic inflammation including high-sensitivity C-reactive protein, ferritin, fibrinogen, P-selectin, and vascular cell adhesion molecule-1. The associations between BM activation and MetS (and its components) and increased erythropoiesis were maintained in the subgroup of participants with no systemic inflammation. Bone marrow activation was significantly associated with high arterial metabolic activity (18F-FDG uptake). The co-occurrence of BM activation and arterial 18F-FDG uptake was associated with more advanced atherosclerosis (i.e. plaque presence and burden). In apparently healthy individuals, BM 18F-FDG uptake is associated with MetS and its components, even in the absence of systemic inflammation, and with elevated counts of circulating leucocytes. Bone marrow activation is associated with early atherosclerosis, characterized by high arterial metabolic activity. Bone marrow activation appears to be an early phenomenon in atherosclerosis development.[Progression of Early Subclinical Atherosclerosis (PESA); NCT01410318].The PESA study is funded by the CNIC and Santander Bank. The present study was partially funded by an intramural grant CNIC-Severo Ochoa to D.S. and B.I. B.I. is supported by the European Commission (H2020-HEALTH 945118 and ERC-CoG 819775). The CNIC is supported by the ISCIII, the Ministry of Science and Innovation, and the Pro CNIC Foundation. CNIC is a Severo Ochoa Center of Excellence (CEX2020-001041-S).S